Binary Embedding: Fundamental Limits and Fast Algorithm

Binary embedding is a nonlinear dimension reduction methodology where high dimensional data are embedded into the Hamming cube while preserving the structure of the original space. Specifically, for an arbitrary N distinct points in S, our goal is to encode each point using mdimensional binary strings such that we can reconstruct their geodesic distance up to δ uniform distortion. Existing binary embedding algorithms either lack theoretical guarantees or suffer from running time O ( mp ) . We make three contributions: (1) we establish a lower bound that shows any binary embedding oblivious to the set of points requires m = Ω( 1 δ logN) bits and a similar lower bound for non-oblivious embeddings into Hamming distance; (2) we propose a novel fast binary embedding algorithm with provably optimal bit complexity m = O ( 1 δ logN ) and near linear running time O(p log p) whenever logN ≪ δ√p, with a slightly worse running time for larger logN ; (3) we also provide an analytic result about embedding a general set of points K ⊆ S with even infinite size. Our theoretical findings are supported through experiments on both synthetic and real data sets.